A Rotate-and-Solve Procedure for Classification

Many high dimensional classification techniques have been proposed in the literature based on sparse linear discriminant analysis (LDA). To efficiently use them, sparsity of linear classifiers is a prerequisite. However, this might not be readily available in many applications and rotations of data are required to create the needed sparsity. In this paper, we propose a surprisingly simple rotation to create the required sparsity. The basic idea is to use the principal components of the sample covariance matrix of the pooled samples or its simple variants to rotate the data first and to then apply an existing high dimensional classifier. This rotate-and-solve procedure can be combined with any existing classifiers, and is robust against the sparse level of the true model. We show that this rotation does create the sparsity needed for high dimensional classifications. The methodological power is demonstrated by a number of simulation and real data examples and the improvements of our method over some popular high dimensional classification rules are clearly shown. ∗Fan’s research is supported by the National Institute of General Medical Sciences of the National Institutes of Health through Grant Numbers R01-GM072611 and R01-GMR01GM100474 and National Science Foundation grant DMS-1206464.